In many computing tasks requiring significant computing power, such as complex calculations or graphical simulations, there is a need to use computer clusters including many computers. The Internet and constantly improved bandwidth has made it possible to access such computer clusters, sometimes referred to as computing centres, remotely, and there is thus a desire to centralize such computing centres, and to locate them where the associated costs are low. Ideal locations should have e.g. a cool climate (requiring less cooling), affordable electricity prices, and low rent. Another advantage of centralizing the computing centres is that is allows increased security.
However, although bandwidth can generally be increased almost infinitively, there is a problem of high latency over long distance connections. This problem is difficult to completely eliminate with current networking technology.
Another factor influencing the requirement on centralized computing centres is the development of remote desktops. Remote desktops were initially developed to provide support to end users and in certain cases, for running a computer aided engineering (CAE) application with its native interface inside a local area network (LAN). The remote desktops allow engineers in different locations to access a single data centre within the organization, and to work from any part of the globe (on a thin client), similar to how they do on their local workstation or within a LAN. With engineering teams of a single organization spread across the globe, and high-end graphics demand of today's CAE applications, it is becoming increasingly difficult to sustain satisfactory performance of such centralized data and computing centres.
Conventionally, a centralized computing centre, e.g. a high performance computing (HPC) centre, is handled by a so called master node, running a resource scheduler and one or several remote desktop applications. This is an inflexible design, lending little possibility to optimize the scheduling of desktop applications.
Further, the centralized computing resources (cluster nodes) in a cluster) are generally heterogeneous, catering to different needs for different applications. Also, applications with different resource requirements often need to interface with each other during run-time. Obviously, the more users that access the same computing centre, the more difficult it gets to share the scarce resources (e.g. graphics capable computers) between them in an efficient manner.
For all these reasons, the centralized computing resources of an organization have traditionally been more geographically dispersed than what is in fact optimal from a cost and efficiency point of view. For example, it becomes increasingly difficult to optimally utilize the centralized computing resources. Sometimes, resources at a computing centre are idling with very few jobs running, while at the same time, there is a high demand for resources at another centre where many jobs are waiting in queue. Although engineers would like to use the idling resources, high latencies and practicalities involved with submitting a job, prevent them from doing this.
Centralized computing centres will also provide capabilities to grow and shrink resources at one central location, in other words, this can lead to the creation of a “CAE cloud”.